Effectiveness of the Kozachenko-Leonenko estimator for generalized entropic forms.

نویسنده

  • Sílvio M Duarte Queirós
چکیده

In this Brief Report we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalized to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either of linear or nonlinear kind). We show that when the variables are independently and identically distributed the estimator is only valuable along the whole domain if the data follow the uniform distribution, whereas for other distributions the estimator is only effectual in the limit of the Boltzmann-Gibbs-Shanon entropic form. We also analyze the influence of the dependence (linear and nonlinear) between variables on the accuracy of the estimator between variables. As expected in the last case the estimator loses efficiency for the Boltzmann-Gibbs-Shanon entropic form as well.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

On the effectiveness of a binless entropy estimator for generalised entropic forms

In this manuscript we discuss the effectiveness of the Kozachenko-Leonenko entropy estimator when generalised to cope with entropic forms customarily applied to study systems evincing asymptotic scale invariance and dependence (either linear or non-linear type). We show that when the variables are independently and identically distributed the estimator is only valuable along the whole domain if...

متن کامل

On the Kozachenko-leonenko Entropy Estimator

We study in details the bias and variance of the entropy estimator proposed by Kozachenko and Leonenko [10] for a large class of densities on Rd. We then use the work of Bickel and Breiman [2] to prove a central limit theorem in dimensions 1 and 2. In higher dimensions, we provide a development of the bias in terms of powers of N−2/d. This allows us to use a Richardson extrapolation to build, i...

متن کامل

Analysis of k-Nearest Neighbor Distances with Application to Entropy Estimation

Estimating entropy and mutual information consistently is important for many machine learning applications. The Kozachenko-Leonenko (KL) estimator (Kozachenko & Leonenko, 1987) is a widely used nonparametric estimator for the entropy of multivariate continuous random variables, as well as the basis of the mutual information estimator of Kraskov et al. (2004), perhaps the most widely used estima...

متن کامل

Efficient multivariate entropy estimation via k-nearest neighbour distances

Many statistical procedures, including goodness-of-fit tests and methods for independent component analysis, rely critically on the estimation of the entropy of a distribution. In this paper, we seek entropy estimators that are efficient in the sense of achieving the local asymptotic minimax lower bound. To this end, we initially study a generalisation of the estimator originally proposed by Ko...

متن کامل

Risk Estimation and Feature Selection

For classification problems, the risk is often the criterion to be eventually minimised. It can thus naturally be used to assess the quality of feature subsets in feature selection. However, in practice, the probability of error is often unknown and must be estimated. Also, mutual information is often used as a criterion to assess the quality of feature subsets, since it can be seen as an imper...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • Physical review. E, Statistical, nonlinear, and soft matter physics

دوره 80 6 Pt 1  شماره 

صفحات  -

تاریخ انتشار 2009